Mosaicing from a Translating Camera, with Eecient Application of Super-resolution

Mosaicing and super resolution are two ways to combine information from multiple frames in video sequences. Mosaicing displays the information of multiple frames in a single panoramic image. Super-resolution uses regions which appear in multiple frames to improve resolution and reduce noise. Combining both methods enables to represent all the static information in the sequence in a compact and visual way. This work presents new algorithms for mosaicing, super resolution, and their combination. In the rst part, a solution is proposed for mosaicing video sequences taken from a translating camera. Based on some general assumptions on the geometrical structure of the scene, the algorithm handles the parallax which is the result of depth di erences, and creates a mosaic image which is tailored to the scene. We call this method Generalized Manifold Mosaicing . Only few of the existing mosaicing algorithms can handle these depth di erences, and it is shown that the proposed algorithm outperforms them. The proposed algorithm can also solve the case of a camera rotating along an arbitrary constant axis. When trying to combine mosaicing and super resolution there is a problem: The alignment which is required for proper mosaicing and the one which is used for super resolution are di erent. This problem is discussed, and a solution is suggested. Finally, e ciency of the super resolution algorithm is very important when trying to apply it to large mosaics. In the second part, existing super resolution techniques are analyzed in the framework of solution of large sparse linear equations. It is shown that the gradient of the function which is minimized when solving these equations can be computed by means of image operations like warping, convolutions, etc. This analysis paves the way for new algorithms, by easily applying known gradient-based optimization techniques for super resolution. As an example, super resolution is implemented with the conjugate-gradient method, and a considerable speedup in the convergence of the algorithm is achieved.